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GETTING THE REQUIREMENTS RIGHT 


It is still too often the case that computer-based application sys- 
tems are developed behind schedule, over cost, do not do as much 
as promised, and do not satisfy their users. After twenty years of 
concentrated attention, why do these troubles continue to arise? 
A good part of the answer to this question is: because the require- 
ments for these application systems were never stated accurately 
and completely in the first place. If the requirement statements 
are erroneous or incomplete, how can the resulting application 
systems be expected to perform satistactorily? At long last, this 
problem area is beginning to receive the attention it deserves. 
Here is what we found in our study of how organizations are 
trying to get the requirements right for their application systems. 


Matter Toys, a division of Mattel, Inc., with 
headquarters in iia2wthorne, California, is a major 
toy manufacturer. Annual sales are in the order of 
$250 million and the company employs about 
10,000 people in peak seasons. The company’s 
data processing is performed on an IBM 370/155 
operating under os. There are 15 programmers 
and 8 system analysts in the management infor- 
mation services department. 

In 1972, Mattel was performing a growing 
number of one-time applications, to the point 
where they decided to look for some software to 
help them. These applications were statistical 
analyses of field survey data. After investigating 
several software systems, they purchased the 
Mark Iv system from Informatics Inc., Canoga 
Park, Calif. 

Then in 1974, they investigated a number of 
data base management systems on the market and 
selected ADABAS, marketed in the United States 
by Software ac of Reston, Virginia. 

As things have turned out, these two software 
systems—Mankk Iv and ApABAS—have been useful 
to Mattel for getting their application system re- 
quirements right. But there have been a few sur- 
prises along the way. 


The Mark Iv system proved to be very useful 
for the one-time applications for which it was ob- 
tained. So the question naturally arose: could 
Mark Iv be used effectively in other situations, to 
ease the development of new application sys- 
tems? Mattel decided to experiment with it. One 
rather extreme case was interesting. 


In this particular case, a manager in a user de- 
partment asked for a new, moderately complex 
application system. The request seemed reason- 
able, the benefits appeared sufficiently attractive, 
so the project was approved. A system analyst 
who had been trained in the use of MARK Iv was 
assigned to the job. The analyst and the user de- 
partment manager defined what the new system 
should do—its inputs, its outputs, and its process- 
ing. The analyst then set up the new system 
quickly, using Mark iv, and began giving the 
manager output reports. Some changes were 
needed, and were easily made with the Mark Iv 
system. Very soon, the system was giving the man- 
ager what he wanted. Overall time to implement 
the system was about one month. 


The system was used by this manager for a 
time, and then use began to fall off. Later the 
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manager was transferred to another department— 
and use of the system stopped altogether. 

Looking back at this project, the people at 
Mattel feel that there was something fundamen- 
tally wrong with how the project was conducted. 
They had no complaint with the software tool; 
Manx Iv provided a means of easily setting up and 
changing an application system. The fault lay, 
they feel, with the trial-and-error approach that 
was used. Yes, it could easily correct minor errors 
in the system design. But it did not get at the basic 
problem. “It cured the itch, not the disease,” they 
said to us. 

Mattel feels strongly that it is preferable for an 
analyst to spend more time on analysis of require- 
ments and design of the new system, with rela- 
tively less time spent by programmers on coding 
and testing. Good software tools can ‘only supple- 
ment the analysis phase, not replace it. 

Now, when a detailed study of a user request is 
authorized, the system analyst is expected to look 
at the problem in an overall context. The analyst 
assumes that the system will be built the way the 
customer has requested it, but still looks to see if 
there is a better way. The analyst is expected to 
make a basic problem identification and defini- 
tion. What is the business problem? What is user 
department management trying to accomplish 
with the new system? Where does the problem 
really lie? Is this a one-time problem or a recur- 
ring one? 

When the analyst feels that he understands the 
problem, he develops a preliminary design of the 
new system. Often, a “typewriter simulation” of 
the new system is used. That is, the analyst types 
up sample output reports, using real data, just as 
would be produced by the new system. The ana- 
lyst goes over these reports with the users, to find 
out how the users will actually use the reports. 
This step usually uncovers more new require- 
ments and more changes in design, we were told. 

With the preliminary design developed, the 
analyst is in a position to estimate development 
costs, operating costs, and maintenance costs. If 
the system is to be a production system, it is de- 
signed to minimize people time, machine time, 
and maintenance time. The choice is made at this 
point as to what software tools will be used— 
MARK Iv, COBOL, ADABAS, or other. 

But no matter how well the analysis has been 
performed up to this point, it is almost certain 
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that changes will be required before the system is 
completed. The people at Mattel ruefully cite 
two typical user comments. Just as programming 
is about to begin, a user says casually, “Oh, by the 
way, can you also give me...” Then, while con- 
version to the new system is underway, a user 
slaps his head and says, “Oh, my God, I am going 
toneed...” 

Both Mark rv and ApaBas provide a good de- 
gree of the flexibility needed to handle such 
changes, we were told. And when production sys- 
tems have been programmed in coBOL, it has 
proved to be reasonably flexible, too, they said. 

With Mark 1v, it is very easy to change report 
formats, add new fields to a report (assuming the 
data is in the file), and so on. Also, a number of 
user departments have learned to use Mark IV for 
setting up their own application systems and de- 
veloping new reports from existing systems. 

With Apasas, it is much easier to add new 
fields to records in the data base than it is to add 
new fields to records in conventional application 
files, say the people at Mattel. With conventional 
files, adding data fields may require changes to all 
or many of the programs in the application sys- 
tem. With Apasas, adding data to files involves 
no modification to the programs that make no use 
of that data; they never see the new data. So the 
changes in requirements that come to light as sys- 
tems are being developed are much easier to 
make with an ApaBas data base. 

Both Mark iv and ApaBas are providing Mat- 
tel with more flexibility for adjusting to changes 
in their application system requirements. 


The need for flexibility 


Just how important is this need for flexibility in 
adjusting to changes in system requirements? 

The user's requirements for a new application 
system should state what the new system should 
do, in order to solve the user’s business problem. 
The most basic requirement, therefore, is that the 
new system address the right problem. 

But even when the problem has been correctly 
identified and is being addressed, it is still possible 
to have errors and omissions in the requirements. 
(No, that statement is not quite right; it is likely 
that there will be errors and omissions in the re- 
quirements.) As the new system is being imple- 
mented, these mistakes come to light and have to 
be corrected. Here are some examples of the 


types of requirements mistakes that we have 
witnessed. 

Missing or incomplete requirements. A particu- 
larly discouraging type of mistake occurs when 
the users and system builders concentrate so heav- 
ily on the main functions of the new system that 
they completely overlook an essential sub-system. 
We remember a case that happened some years 
back in which a manufacturing company was de- 
veloping a new computerized production control 
system. The project team was well into pro- 
gramming when someone asked why no financial 
data was being collected by the system. It was 
suddenly realized that the whole financial sub- 
system had been overlooked. Most oversights are 
not of that magnitude, of course, but they still can 
cause considerable rework. 

When the users begin to get output reports and 
documents from the new system, the need for ad- 
ditional data fields often becomes apparent. This 
generally means new data fields have to be picked 
up on input, carried in the records, and produced 
on the outputs. The reason for the oversight may 
be that the new system is doing things quite 
differently from the old system and the need for 
those data fields was not recognized. Associated 
with these data fields would be the logic for han- 
dling the new data, which might involve some 
subtle complexities. Adding the data and the logic 
can be difficult after the system structure has been 
established. 

Other types of oversights include timing con- 
siderations (when things must happen), security 
and internal control requirements (which we dis- 
cussed last month), terminal operating needs for 
on-line systems, and testing requirements for the 
new system. 

Ambiguous requirements. Ambiguous require- 
ments are ones where the user’s interpretation 
differs from that of the system builders, due to im- 
precise statements. One area of difficulty is the 
transition period when converting from the old 
system to the new one—and what to do with the 
transactions that are in the pipeline during this 
period. The user may assume that the new system 
will take them over, while the system builders 
may assume that those transactions will be com- 
pleted by the old system. Another problem area 
occurs when a point in time is treated as if it were 
a time period, or vice versa. Thus, “the first of the 
month” might have to be defined as 12:01 a.m. on 
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the first day of the month, and not the whole first 
day. 

Requirements errors such as these are common. 
As they come to light, the new system has to be 
changed to accommodate them. If the new sys- 
tem has been implemented with flexible software 
tools, the changes are much easier to make. We 
have already mentioned ApaBas and Mark Iv. 
Comparable software tools include System 2000 
and AsI-sT, to name two—and, of course, there 
are numerous other packages that should be 
considered. 

With software packages such as these, it is usu- 
ally easy to define new data fields and record 
types, and it is easy to change those definitions. 
Data fields can be added or deleted, lengthened or 
shortened, and so on. When such data definition 
changes are made, only those programs that use 
those data definitions may need to be changed; 
other programs accessing other parts of the data 
files are unaffected. Powerful retrieval and output 
formatting languages are available, for rapidly 
creating the programs for output reports and 
documents. So it is often possible to define a data 
file, load the data, and create user reports very 
quickly, sometimes in a matter of hours. And if 
the user wants changes made in these outputs, 
these can be made quickly, too. 

Let us go back to the question of mistakes in the 


user requirements. Just how serious a problem is 
this? 


The problem with requirements 


The question must be asked, of course: do user 
requirements really represent a difficult problem 
in the development of application systems? Some 
people say No, the requirements are not the prob- 
lem. The users simply state their requirements; 
the trouble is that the implementors are not able 
to meet those requirements. The main difficulties 
lie not with the users but with the implementors, 
say these people. | 

This report will support the viewpoint that the _ 
user requirements are the source of much of the 
difficulty. Typically, requirement statements are 
filled with a variety of errors, as described above. 
When there are errors in the requirements state- 
ments, it is unreasonable to expect that the result- 
ing application system will be error-free. 

The Second International Conference on Soft- 
ware Engineering, sponsored by the Association 


for Computing Machinery, by the Computer So- 
ciety of the Institute of Electrical and Electronic 
Engineers, and by the U.S. National Bureau of 
Standards, was held in San Francisco, California 
in October 1976. A series of sessions at this confer- 
ence considered the problem of system require- 
ments and different solutions for getting the 
requirements right. Reference 1 is the 
proceedings. 

In his paper presented at this conference, Har- 
lan Mills gave the rationale (we believe) of why 
some people feel that requirements are not a 
problem. With manual systems, said Mills, the 
people who operated the systems used common 
sense to make the systems work. Managers could 
state vague requirements; over a period of time, 
the people operating the systems would make 
necessary adjustments to compensate for the vague 
statements of requirements would be suitable for 
agers have expected that the same type of vague 
statements of requirements would be suitable for 
computer-based systems. But that is not the case. 
Computers have no common sense; they just fol- 
low orders. So faulty or missing instructions can 
wreak havoc, said Mills. Getting complete and 
accurate statements of requirements for com- 
puter-based systems is a problem. 

What is wrong with the conventional approach 
for stating requirements for computer-based sys- 
tems? D. Teichroew and E. A. Hersey, at this 
same conference, reported that they had re- 
viewed the literature covering many approaches 
to system studies and had found no agreement on 
the phases of a development project. Each organi- 
zation used its own methods and standards. There 
was a consensus pattern, however. After a project 
was requested and initially evaluated, a detailed 
study would be authorized. A senior system ana- 
lyst (generally) would study the existing system, 
develop the requirements for the new system, and 
then lay out the preliminary design of the new 
system. The users would then be asked to review 
this preliminary design and to indicate all neces- 
sary changes. Out of this would come the system 

requirements report that would provide the basis 
for the detailed construction of the system. By 
studying a number of these system requirements 
reports, Teichroew and Hersey reported that the 
average report consisted of about 30% text in nat- 
ural language, 50% in lists and tables, and 20% in 
graphs, flowcharts, and drawings. But natural lan- 
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guage is not sufficiently precise for stating re- 
quirements; further the reports tend to be 
voluminous, it is hard to assure consistency, and it 
is hard to tell if something is missing. In addition, 
continual change makes these problems worse. 

Mills added another point to the difficulties en- 
countered with the conventional approach. The 
user department people who really understand 
the present system—and who are in the best posi- 
tion to state requirements for the new system—are 
usually too busy to participate. So surrogate ex- 
perts with more time available (guess why!) are 
assigned to the projects by the user departments. 
These surrogate experts give amateur opinions, 
which lead to missing and incorrect requirements. 

D. T. Ross and K. E. Schoman, Jr. (Reference 1) 
pointed out other weaknesses with the conven- 
tional approach. One weakness is the inability of 
the people on the project to clearly see what the 
problem is, much less to measure it and visualize a 
workable solution. This is because the complexity 
of the application makes it difficult to understand. 
Another weakness is that the project members 
tend to think of the system architecture in terms 
of devices, languages, record formats, and so on, 
rather than thinking of the solution of the basic 
problem. 

M. N. Jones (Reference 6) points out another 
difficulty with the conventional approach. The 
user communicates with the analyst who in turn 
may communicate with the system designer who 
in turn communicates with the program designer 
who then communicates with the programmers. 
Misunderstandings can and do occur at each level 
of interface, she says. Also, users miss good op- 
portunities for getting more value from the sys- 
tems because they are too busy to participate to 
any great extent and they do not understand what 
the computerized system can do for them. 

What do all of these weaknesses result in? T. E. 
Bell and T. A. Thayer (Reference 1) report on 
studies they have made on requirement errors. 
Two projects were studied in detail, where it was 
possible to keep track of the errors as they were 
uncovered. The types of errors, as they occurred 
on these two projects, were: (1) incorrect require- 
ments, (2) missing/incomplete/inadequate re- 
quirements, (3) unclear/ambiguous requirements, 
(4) inconsistent/incompatible requirements, (5) 
new/changed requirements, (5) requirements 
outside of the scope of the project, and (6) typo- 


graphical errors in the requirements statements. 
On the smaller of the two projects, the require- 
ments statements consisted of 48 pages of text and 
charts; by the end of the design phase, over 50 er- 
rors had been found, more than one per page. On 
the larger project, the requirements statement 
consisted of some 2,500 pages with over 8,000 
uniquely identifiable requirements. Two major re- 
quirements reviews were conducted and these 
uncovered almost 1,000 uniquely identifiable 
problems, some serious enough to lead to the fail- 
ure of the system to perform its mission. 

One can conclude that it is very difficult to 
state requirements accurately and completely. If 
errors are likely to exist in requirements state- 
ments, what is the impact of these errors? 


The impact of requirements errors 


W. W. Black (Reference 1) points out that the 
things that put a development project behind 
schedule are the things that were not on the task 
list. The items on the task list are generally done 
close to schedule, he says. Incorrect and unclear 
requirements can lead to rework; missing and in- 
complete requirement statements mean that 
items must be added to the task list. So errors in 
the requirements statements might well be re- 
sponsible for a good part of the schedule slippages 
that have occurred in development projects. 

B. W. Boehm, in remarks given at the Software 
Engineering Conference, commented on the cost 
impact of errors in requirements. From the study 
of four projects, it was found that the relative cost 
of correcting an error increases exponentially 
with the project phase in which the error is de- 
tected. A requirements error that is not found un- 
til the testing phase can cost 10 to 100 times as 
much to fix as it would cost if it had been found 
during the specification phase. Further, conven- 
tional thinking says that perhaps 40% of devel- 
opment costs occur during the design phase, 
another 20% occur during coding, and the re- 
maining 40% occur during testing. Looking at the 
actual costs over the life cycle of a system, these 
figures just are not right, says Boehm. Instead, de- 
sign represents only about 12% of the costs, cod- 
ing 6%, testing 12%—and maintenance 70% of the 
costs. Much of this maintenance is due to not get- 
ting the requirements right in the first place, he 
says. 

So the rework of systems, due to errors in the 
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requirements, leads to schedule slippage, cost 
overruns during development, and continuing 
maintenance costs. We are not saying that only 
the requirements errors lead to these; certainly 
errors are injected during design and construc- 
tion. But the requirements errors are fundamental 
and appear to have a large impact. 

In addition, requirements errors undoubtedly 
lead to user dissatisfaction with the application 
systems. Computer-based systems are not self-ad- 
justing as are manual systems, as Mills points out. 
If the requirements are not stated correctly, then 
the systems are not going to perform to the user’s 
satisfaction. 


Conclusions about requirements errors 


Requirements errors are a real and a serious 
problem. Further, they are quite difficult to pre- 
vent and to eliminate. The only safe assumption 
to make is that any conventional requirements 
statement (and resulting system specification) is 
filled with errors. Steps must be taken to detect 
and remove those errors as soon as possible. 

Requirements errors can and do adversely af- 
fect a project in terms of schedule, costs, and sys- 
tem performance. 

It is clear that much more attention must be 
paid to “getting the requirements right.” 


New methods for determining requirements 


From our study of this subject, we see a seven 
element program for reducing the number of er- 
rors in application system requirements. These 
are the seven elements: 

¢ Awareness of the types of errors 

¢ User involvement 

¢ Ways to handle complexity 

¢ Formal inspection procedures 

° Definition of performance 

¢ Formal language for requirements 
Automated tools for analysis 
We will discuss each of these briefly. 


Awareness of types of errors 
T. E. Bell and T. A. Thayer (Reference 1) have 


analyzed the types of errors found in require- 
ments statement documents, as we mentioned 
earlier in this report. The analysis was made on 
one relatively small project, conducted in a uni- 
versity environment, as well as on a large military 
project. More recently, they have checked the re- 


sults found from these first two projects on a third 
project and have encountered reasonably similar 
results. 

The relative percent of errors in requirements 
found on these projects was as follows: 


% of Total 

Errors 
Incorrect requirements 34% 
Missing/incomplete/inadequate 24% 
Unclear/ambiguous 22% 
Inconsistent/incompatible 9% 
New/change 3% 
Outside scope of project 4% 
Typographical errors 4% 


Also as we indicated earlier, Bell and Thayer 
found an average of between one-half and one re- 
quirements errors per page of detailed require- 
ments statements. | 

The point to be made here is that data process- 
ing management should recognize that a substan- 
tial number of errors will exist in most 
requirements statements unless specific action is 
taken to identify and remove them. The percent- 
ages found by Bell and Thayer may not apply in 
any particular other situation; we will discuss be- 
low how each organization might develop its own 
figures. But these figures do give some idea of 
where the worst part of the problem probably 
lies—in incomplete, missing, and unclear 
requirements. 

It is true that errors can be introduced in the 
later stages of a project, such as in design, con- 
struction, or even during testing. Two comments 
are appropriate. The earlier the errors are in- 
troduced and the longer it is before they are 
caught, the worse the impact of them will gener- 
ally be. So requirements errors might be consid- 
ered to be the worst kind. Also, the methods 
proposed below for catching requirements errors 
can also be useful for catching errors introduced 
during the later stages of a project. 

Thus if both general management and data 
processing management recognize that require- 
ments statements are error-prone and that these 
errors lead to later difficulties, the stage is set for 
effective corrective action. 


User involvement 


It is trite to say that user management should 
be involved in the development of requirements— 
but, as the saying goes, “trite is right.” 
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It simply is not adequate for user department 
management to spend only an hour or two with 
the system analyst(s), telling what is wanted in the 
new system, and then expect the system to meet 
their needs and desires. It also is not adequate for 
user department management to turn this respon- 
sibility over to a staff person. 

A consultant friend of ours, whose firm has au- 
dited many system development projects, said 
that he thought that user management ought to 
think in terms of a two-day session for developing 
requirements for a typical business application 
system. This amount of time matches our expe- 
riences in the past, in developing system needs. 
But even this amount of time is not sufficient for 
determining all of the requirements. So “review 
sessions’ or “walk-throughs”’ or “inspections’’ are 
needed in the specifications and design phases. In 
such sessions, which are usually two hours or so in 
length, user management can catch additional re- 
quirements errors and omissions. 


Ways to handle complexity 


Complexity is perhaps the root cause of re- 
quirements errors. This applies to both appli- 
cation systems and generalized software systems, 
such as operating systems. It is too often the case 
that such systems have too many details for one 
person to fully comprehend. 

The point here is that unless the person(s) pre- 
paring the requirements statements fully under- 
stand the system under study, errors will be 
introduced. Some requirements will be omitted, 
some will be stated incorrectly, some will be un- 
clear, and so on. 

So the question is, how can the users and ana- 
lysts gain understanding of what the system must 
do and must not do, in the face of the complexity 
that exists? 

Here are several approaches that have been 
used successfully for gaining this understanding in 
the presence of complexity. 

Progressive approach. The progressive ap- 
proach is the “divide and conquer” approach. We 
discussed it in our October 1970 report and in sev- 
eral subsequent reports. The concept is that a big 
project is sub-divided into a number of smaller, 
stand-alone projects, none of which requires more 
than six or nine months to accomplish. Also, it is 
better if each project is done by a team of not 
more than three to six people. 


Complexity will be reduced by tackling the big 
project as a series of relatively small steps. 

It probably will be desirable to have at least a 
preliminary design for the overall system, so that 
each sub-project will integrate reasonably well 
with previous sub-projects. We have seen this 
done. It is hard to say if it can be done effectively 
in all cases—but we suspect that it can be used 
much more widely than is currently true. 

Top-down analysis. One concept that can be 
used is described in IBM’s Study Organization 
Plan (Reference 4). Start with an overall view of 
the organization—its products or services, its mar- 
ket, its competitors, the resources available, and 
so on. Then divide the organization into goal- 
oriented activities—cohesive groups of functions 
that aim to satisfy a specific part of the organiza- 
tion’s market. The goal-oriented activities, in 
turn, are sub-divided into their specific com- 
ponent operations, data, data flows, and so on. 

The concept of this particular form of top- 
down analysis is that understanding will be in- 
creased by viewing the specific operations per- 
formed by people in the organization in terms of 
what the organization is trying to accomplish in 
its marketplace. (And whether the organization is 
commercial, governmental, educational, or other, 
it does have a market for its products or services.) 

Another currently-popular concept of top- 
down analysis has been given names such as func- 
tional decomposition or levels of abstraction. We 
discussed levels of abstraction in connection with 


structured programming, in our June 1974 issue. 


The idea is that the analyst first takes a broad view 
of the overall function to be performed. The ana- 
lyst tries to identify all of the inputs, all of the re- 
sources, and all of the outputs that are 
appropriate for this top level view. In addition, 
the analyst attempts to list all of the component 
sub-functions that make up this top level 
function. 

The process is then repeated for each of the 
sub-functions at the second level. Each sub-func- 
tion is analyzed in terms of its inputs, resources, 
outputs, and component sub-sub-functions. By 
concentrating on one function at a time and de- 
ferring any lower level details for later consid- 
ering, understanding is enhanced. 

What results is a hierarchy of functions. IBM’s 
H1Po and SofTech’s sapT use this approach of the 
successive decomposition of a complex function 
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into a hierarchy of understandable sub-functions. 

Information flow and data analysis. This ap- 
proach to the handling of complexity might be 
considered the opposite of the functional decom- 
position method just described. It aims at giving a 
complete picture of the function being studied, in 
both breadth and depth. 

Information flow analysis of a function would 
be shown in flow diagram form; the process is de- 
scribed in Reference 3. For a large, complex func- 
tion, the flow diagram might be very large, 
covering one or more walls of a room. Boundaries 
of existing applications within the overall func- 
tion might be indicated on the diagram, thus in- 
dicating how the different applications tie 
together to make up the overall function. 

Data analysis might be documented by a series 
of data definitions, tied to the information flow 
analysis. We saw one instance where the data def- 
initions were written on small cards and attached 
to the walls of the study room, grouped by the ap- 
plications within the overall system. 

Information flow and data analysis are not as 
popular today as is, say, functional decomposi- 
tion. A little later in this report we will discuss 
some of the advantages and shortcomings of each 
of these methods. Suffice it to say here that infor- 
mation flow and data analysis have been used suc- 
cessfully for handling complexity. 

Iteration/convergence. This approach to the 
handling of complexity assumes that it is not pos- 
sible to get all of the requirements right for a com- 
plex system before that system is built. This 
assumption holds even if one or more of the 
above-described methods of handling complexity 
are also used. This approach uses tools which 
make it easy to construct the new system, get it 
running, and to change it when errors are uncov- 
ered. By successive refinements of the system, it 
gradually converges toward the system that the 
users desire. 

It is important for the system development or- 
ganization to select some method for handling 
complexity. Actually, several of the above- 
described methods might be used in combination. 
There is no reason why the progressive approach, 
top-down analysis, and iteration/convergence 
cannot all be used. In fact, if we interpreted his 
remarks correctly, that was just what Harlan 
Mills was advocating at the Software Engineering 
Conference mentioned earlier. 


Formal inspection procedure 


B. W. Boehm, in his remarks at the Software 
Engineering Conference, reported on studies that 
showed inspections and walk-throughs were the 
most effective means for catching errors early. 
Formal requirements languages, standards, simu- 
lation, and automated tools were less effective 
(but still useful) for catching the errors. 

Formal inspection procedures would thus seem 
to be mandatory if data processing management 
truly wants to get rid of requirements errors. 

Formal inspection does have a connection with 
a project management system. Project manage- 
ment systems generally specify formal check- 
points during the phases of a project, as well as 
the work products that must be completed before 
each checkpoint. Project progress is measured 
when the checkpoints have been successfully 
passed. During a checkpoint review, manage- 
ment might be looking for any significant changes 
in project costs, schedules, or expected benefits. 
As far as the work products are concerned, the 
main question of the review is: “Have each of the 
required work products been completed?” 

Formal inspection goes one step further. It 
asks: ““What is the quality of those work 
products?” 

M. E. Fagan (Reference 2) argues that formal 
inspections are more effective than walk- 
throughs. In a walk-through, someone (say, a pro- 
grammer) describes to a selected, small group of 
people the flow of control and the handling of 
data in a program being reviewed. The group of 
people is expected to question the person and 
point out possible flaws, possible omissions, and so 
on. But a walk-through is largely an educational 
process, the participants get sidetracked into dis- 
cussing design alternatives, and the process is not 
self-improving, says Fagan. 

Fagan discusses his inspection procedure in 
terms of the inspection of program design and 
coding. It can also cover test planning and execu- 
tion, documentation, rework, and other activities. 
We see no reason why it could not also be used at 
the end of the requirements and specification 
phases, for catching errors as early as possible. 

Fagan’s inspection program has five parts. (1) 
First is a short overview of the work to be in- 
spected, presented by the analyst or designer who 
did the work. This is the educational, familiar- 
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ization step. (2) Next, each of the inspection team 
participants is given copies of the documentation 
and is expected to do “homework” on it. Each one 
tries to obtain an understanding of the work by 
studying the documentation. Some errors may be 
caught at this point, but not the bulk of them, says 
Fagan. (3) The third step is the inspection meet- 
ing itself. The goal of this meeting is to find errors. 
It is up to the moderator to see that the discussion 
does not get sidetracked into, say, considering al- 
ternative designs. There should be no hunting for 
solutions to the errors. Just find the errors, says Fa- 
gan—and classify them by type and estimate their 
severity. Following the inspection meeting, the 
moderator is expected to write up an inspection 
report, listing all of the errors. (4) The next step is 
the rework of the work, to get rid of the errors. (5) 
Finally, at least the moderator (and perhaps the 
whole team) must perform a follow-up to see that 
all fixes have been made and made properly. De- 
pending upon the number and severity of the er- 
rors, another inspection meeting by the whole 
team may be required. 

In addition to the error report, there are several 
other outputs of this inspection procedure, says 
Fagan. One important output is the building up of 
a' checklist of error types, their frequency of oc- 
currence, and their severity. Coupled with this 
checklist will be a procedure of how to look for 
errors, particularly those that occur most fre- 
quently and/or are most severe. Because formal 
inspection reports are prepared, the inspection 
process itself can be analyzed, to see how it can be 
improved. Not only can these outputs be used by 
future inspection teams to improve inspections, 
they can also be given to individual analysts, de- 
signers, and programmers to show them what er- 
rors to guard against in the future. 

Fagan cautions data processing management 
on this last point. Do not use these inspection re- 
ports for employee performance appraisal, he 
says. If they are used in performance appraisal, 
then the employees will look on inspection meet- 
ings as a threat rather than as an aid. They will 
fight the system instead of supporting it. 

What about the inspection team? It should be a 
small team, of perhaps four people, says Fagan. 
The moderator is the key person and the success 
of the inspection process is very much dependent 
upon this person. He or she should be competent 
in the subject area but not necessarily in the par- 
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ticular application being reviewed. The moder- 
ator must keep the whole inspection process 
moving along and not let it get sidetracked. 

The other team members might be (for a pro- 
gram that is being inspected) the designer, the 
coder, and the tester of that program. If one per- 
son does two or three of these functions, then find 
qualified substitutes for the other one or two spots 
on the team. If the program interfaces with other 
programs, then the programmers of those other 
programs can be on the team. As we say, Fagan 
discussed the procedure in terms of the inspection 
of programs. A comparable team can be vis- 
ualized for inspecting the requirements for a new 
system. 

It seems to us that a formal inspection proce- 
dure of this type is mandatory in any management 
program designed to detect requirements errors. 
As Boehm reported, it appears that inspection is 
the most effective single way for detecting and 
eliminating errors. 


Definition of performance 


There are really two distinct points to be dis- 
cussed here. 

Define performance of the present system. If 
management expects that the performance of the 
new system will be “better” than that of the sys- 
tem it replaces, some baseline is needed. The way 
to get this baseline is to measure the performance 
of the present system. 

IBM’s Study Organization Plan (Reference 4) 
provides a convenient method of documenting 
the performance of the existing system. The per- 
formance measures include the volume of trans- 
actions handled, the elapsed time for handling a 
given transaction, the man-hours required for the 
different operations, and so on. 

Analysts frequently object to studying the pres- 
ent system in detail because “it is wasted time 
since the new system will do things differently.” It 
is true that some of the “how” of the present sys- 
tem will be useless, as far as the new system is con- 
cerned. But the “what” and “how many” and 
“how long” of the present system are meaningful. 

Develop performance evaluation tests for the 
new system. Qualitative requirements statements 
generally are unsatisfactory, such as “we desire 
more flexibility in adapting to changes in cus- 
tomer orders.” Any such statements are unclear. 


They should be replaced by specific statements of 
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what performance is expected of the new system. 

M. Alford, at the Software Engineering Con- 
ference (Reference 1), pointed out that when a 
person attempts to write the validation tests for a 
performance requirement, he is forced to ask 
questions about that requirement and begins to 
uncover errors or omissions in the requirements 
statement. 


To uncover requirements errors, every require- 
ment should be stated in measurable terms and a 
test should be written to validate that require- 
ment. That may not be easy—but it probably is a 
lot better than all of the rework and unhappiness 
that comes from building systems wrong due to 
requirements errors. 


Formal language for stating requirements 


Natural language is not well suited for stat- 
ing requirements. Since natural languages allow 
for imprecision, there can be gaps, loose ends, 
and misunderstandings in the requirements 
statements. 


A formal language for stating requirements has 
the advantage of precision. It is a constrained lan- 
guage, without all of the flexibility of natural lan- 
guage and has carefully defined constructs. When 
a series of requirements statements is made in a 
formal language, the likelihood is much greater 
that they will not be misunderstood, as compared 
with the same statements in natural language. 


Formal languages for requirements statements 
can be in three forms. A narrative language uses a 
concise sentence structure and a defined vocabu- 
lary. Examples are the problem statement lan- 
guage (PSL) developed at the Ispos Project at the 
University of Michigan (Reference 7), and the re- 
quirements statement language (RsL) developed 
at TRW (described briefly in Reference 1 and in 
more detail in Reference 9). A graphic language 
uses charts that must be developed according to a 
strict discipline. Examples are IBM’s H1po system 
(Reference 6) and SofTech’s sapt (Reference 5). 
Finally, a tabular language expresses require- 
ments in table form. Examples of this method in- 
clude IBM’s sop (Reference 4) and NCR’s aps 
(Reference 8). 


The use of a formal language by itself can be 
helpful, by encouraging more accuracy and com- 
pleteness in the statement of requirements. In ad- 
dition, if a narrative formal language is used, it is 


possible that it can also be used with automated 
analysis tools. 


Automated tools for analysis 


Two sets of automated analysis tools were 
briefly reviewed at the Software Engineering 
Conference mentioned above—the problem state- 
ment analyzer (psa) of the University of Michigan 
and the requirements engineering and validation 
system (REVS) developed by TRW. 

If the requirements have been stated in a for- 
mal narrative language, these statements can be 
input to the automated analysis tools. These com- 
puter programs can then help to detect such 
things as gaps in the information flow, all uses for 
each data item (including unused data items and 
conflicting uses for the same data item), conflict- 
ing names, and so on. In addition, they can be 
used to prepare management summary reports, 
for indicating how much of the overall require- 
ments statements have been completed. 


The debate on methodology 


Getting the requirements right is a difficult and 
complex problem. As might be suspected, there 
is no clear consensus on how best to solve this 
problem. 

As the above discussion has hinted, there are 
some significant differences of opinion on how to 
go about stating requirements. We would like to 
briefly review some of the chief schools of 
thought that we encountered in our study. 


Hierarchy of functions vs. information flow 


As we discussed earlier, the decomposition of 
functions and sub-functions into a hierarchy is 
proposed by some as the best way to handle com- 
plexity. Examples of this approach include IBM’s 
H1PO, SofTech’s sapT and Dijkstra’s levels of ab- 
straction method of structured programming. 
This approach concentrates on one level at a 
time, until it is fully understood, before moving 
on to the next lower level of detail. 

At the other extreme is the information flow 
analysis, that traces the flow of all inputs through 
the system, until all outputs have been produced. 
The idea here is that the project team people can 
see the whole system, in its breadth and depth, 
and thus gain an understanding of it. 

The adherents of functional decomposition ar- 
gue that the information flow analysis can lead to 
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huge charts which are too big for any one person 
to grasp mentally. And because of the size of the 
charts, it is very difficult to see errors of omission 
and commission. It is much better, say these 
people, to analyze one level of function at a 
time so as to minimize the errors of omission or 
commission. 

The adherents of information flow analysis (for 
example, see Alford in Reference 1) point out 
some weaknesses of functional decomposition. It 
is hard to trace the flow of control for one input 
message through the hierarchy of functions. Also, 
it is hard to construct test cases for one specific 
sub-function in the hierarchy. Particularly in a 
real-time system, it is important to follow the 
flow of control for each type of input message, to 
make sure that the time constraints will be met 
under all expected conditions. 

We have seen both methods used successfully 
but as yet have no general guidelines of when to 
use and when not to use either method. We 
are simply pointing out here that, while both are 
useful for handling complexity, each has its 
shortcomings. 


Exhaustive study vs. interative/convergence 


These two approaches might be sub-titled “do 
it right” versus “do it over.” 

On one side of the debate are the people who 
say: when you undertake a large system project, 
you must do an exhaustive requirements study—so 
that you do not find half-way through the project 
that you are doing something fundamentally 
wrong. It is just too expensive in time and money 
to have to change directions in the middle of a big 
project. 

On the other side of the debate are the people 
who say: never undertake a big project in the first 
place. Divide a large project into manageable, 
smaller sub-projects. Moreover, build these sub- 
project systems in a manner so that they are easy 
to change. 

However, as we shall see shortly, this statement 
of the two sides of the debate does not really ex- 
press what is being debated. To set the stage for 
what is being debated, let us first consider the “do 
it over” approach. 

In his remarks at the Software Engineering 
Conference, Harlan Mills claimed that human 
ambition (perhaps coupled with some of the re- 
quirements statement techniques discussed at the 
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conference) tend to promote large projects. In 
fact, he said, ambition tends to promote very 
large projects, grandiose requirements, and 
grandiose specifications, where it can take several 
years just to develop the specifications. This is not 
the way to go, he said. Instead, undertake only 
smaller projects that can be managed effectively. 
Break a big project into a number of smaller ones. 
“We can conquer the world, three to six program- 
mers at a time,” he said. 

With the smaller projects, the new systems can 
be set up quickly, the users can be given some out- 
puts, any changes requested by the users can be 
made quickly, new outputs are delivered, and the 
cycle is then repeated. Since changes are rela- 
tively easy to make, no attempt is made to obtain 
an exhaustive set of requirements before the sub- 
project system is designed. 

This, then, is the iterative/convergence ap- 
proach. It divides large projects into smaller ones 
and uses an iterative method to converge on the 
system solution that the user desires. 

The “do it right” proponents do not disagree 
with the concepts of this iterative/convergence 
approach. The problem is, they say, that it is often 
not clear just how it can be used with a big system. 
User management may not be able to see how the 
overall system can be divided, in order to build 
one part of the system at a time and yet be able to 
integrate those parts later. It is difficult to foresee 
at the beginning of a large project all of the ways 
that the multiple parts will impact one another. 
These interactions of the parts can be very subtle. 
It may be necessary to do a detailed analysis of re- 
quirements for the whole system in order to be 
able to sub-divide the total system intelligently. 

So, say the proponents of the exhaustive study, 
we do not disagree with the idea of sub-dividing 
large projects and implementing the parts 
quickly. But on large, complex system projects, 
we just do not know how to use that approach in 
its entirety. We feel that we must often make an 
exhaustive study of requirements before we can 
do the sub-dividing. 

Another argument against the “do it over” ap- 
proach is that the user may see only the effects of 
a basic problem and not the cause of that prob- 
lem. So the user conveys, and the analyst accepts, 
a superficial analysis of the problem. Trial-and- 
error problem solving is not the way to approach 
a complex, challenging problem. The first itera- 
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tion may be so far wide of the mark that con- 
vergence to a good solution does not occur. 
Moreover, this approach tends to promote lazi- 
ness and sloppy thinking on the part of the ana- 
lysts. Also, users are already too inclined to 
change their minds about what they want in ap- 
plication systems; this approach just amplifies 
such vacillations. So say the proponents of the ex- 
haustive study approach. 

We think that both sides make valid points. If 
the application system under consideration is not 
particularly large, then sufficient analysis of the 
requirements must be done to make sure that the 
right problem is being worked on. User manage- 
ment needs must be determined, by obtaining 
user involvement at the appropriate times. Then 
the iterative/convergence approach can be used. 

On the other hand, if the system under consid- 
eration is basically a huge one—for example, as 
was the first computerized airline reservation sys- 
tem—then it may very well be necessary to make 
a thorough study of requirements before sub-di- 


‘viding and implementing. The same comment 


can be made about a generalized application sys- 
tem that is supposed to serve tens or hundreds of 
user organizations, of the type that we discussed 
in our January 1977 report. 


How to get the requirements right 


What is a no-frills program that an organiza- 
tion can use to get the requirements right for new 
application systems and for major revisions to ex- 
isting systems? Following is what we see as a min- 
imum program. 


Recognize the types of errors 


The first step, it seems to us, is to recognize that 
requirements statements will normally have er- 
rors, and lots of them. Moreover, the longer it 
takes to detect these errors, the more serious are 
the time and cost penalties for correcting them. 

Somehow, this fact must be communicated to 
all levels of management in the organization. This 
probably will not be an easy task. Further, in all 
likelihood, this message will not be well received. 
Top management and user department manage- 
ment may think that stating requirements is 
easy—and that if errors occur, they are the fault of 
the data processing people. It may not be easily 
accepted by these managers that many errors are 
likely and they are just as much the responsibility 


1] 


of the user departments as they are of the data 
processing people. 


As we will mention below, a formal inspection 
procedure should be a part of even a no-frills pro- 
gram. As errors are detected, a list of error types 
will be developed. This list would seem to be the 
best possible argument to convince management 
about the errors that normally occur in require- 
ments statements. 


Get user involvement 


As the list of types of requirements errors 
grows, it should become much easier to obtain the 
necessary involvement of user management. 
They can see how these errors in the past have 
caused project delays and added costs. 


Even before this list is available, however, an 
attempt should be made to get the key managers 
(not just staff members) of the user departments to 
spend two full days or so in a “requirements ses- 
sion. This may not be easy to do, and it certainly 
requires the support not only of those managers 
but also of their superiors. But it can be done and 
the results are valuable indeed. We used this ap- 
proach on numerous occasions some years back 
and it always provided very useful results. 


Select an approach for handling complexity 


Earlier in this report, we discussed a number of 
ways for handling complexity. All have worked 
well in specific instances. Here are our prefer- 
ences for a no-frills program. 


Progressive approach. Do everything in terms 
of small, short term projects which are done by 
small groups of people (3 to 6 people maximum) 
in a short period of time (6 to 9 months max- 
imum)—but integrated into an overall program. If 
a big application is to be implemented, develop a 
master design, divide into a series of small proj- 
ects, and develop these sub-systems individually. 


Functional decomposition or information flow 
analysis. We do not have strong opinions that fa- 
vor one of these approaches over the other. Cur- 
rently, the functional decomposition approach is 
receiving a lot of attention, one reason being the 
wide exposure of IBM’s Hipo method. But some 
means is needed for analyzing and documenting 
what the new system must do, what the perform- 
ance of the present system is, and what the per- 
formance of the new system should be. 
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Particularly if the functional decomposition 
approach is used, a formal (graphical) language 
for stating requirements will be involved. It may 
also be desirable to translate from this graphical 
language to a narrative language, such as PSL, in 
order to use automated analysis tools. We see this 
as an optional step, depending upon the size and 
complexity of the total project (covering all sub- 
projects), and not necessarily a part of a no-frills 
program. 

Tools to make changes easier. The types of 
changes that will be involved include adding new 
data fields to the files so that they can in turn be 
added to reports, changing field lengths, changing 
relationships among data items, changing report 
formats, and changing processing logic. Expe- 
rience has shown that some of the file manage- 
ment and data base management systems offer 
more flexibility for such changes than do conven- 
tional programming and data handling methods. 
Some people might disagree and say that conven- 
tional methods are sufficiently satisfactory—but 
we feel that some of these new tools are properly 
a part of a no-frills programz 


Formal inspection program 


As B. W. Boehm has pointed out, it appears that 
a formal inspection program (or the less formal 
walk-through program) is the most effective 
single method for detecting errors in require- 
ments and design. So such a program should cer- 
tainly be a part of a no-frills methodology. 

M. E. Fagan has described a formal inspection 
program that has worked effectively at IBM. Oth- 
ers might take a somewhat different approach. 
We would think that Fagan’s approach would be 
a good way to start an inspection program. 


Define expected performance 


Another powerful way for catching require- 
ments errors is for the analyst to develop perform- 
ance validation tests for all stated requirements. 

This step does a number of things. For one 
thing, it identifies qualitative requirements. Such 
requirements either should be converted into 
quantitative requirements or, failing that, elimi- 
nated. Also, it detects all open ended require- 
ments which are only partially defined. Open 
ended requirements are frequently stated in terms 
of examples, and it is left up to the “common 
sense’ of the implementors to fill in all of the miss- 


12 


ing cases; this usually leads to troubles. This step 
also forces the analyst to determine just what con- 
stitutes good performance and what is unaccept- 
able performance, for each of the requirements. 
As Alford commented at the Software Engineer- 
ing Conference, trying to write the performance 
validation tests really points out what a person 
does not know about the requirements. 


Conclusion 


Progress is being made in the application of 
computer technology—but the problems of 
schedule slippages, cost overruns, and dissatisfied 
users still arise. Because of these problems, the 
data processing function has often lost credibility 
in the eyes of top management; management just 
could not depend upon the promises that were 
made. We believe that the situation is improving, 
as data processing departments have decided to 
limit themselves to smaller projects and not to 
promise as much. But the situation, while improv- 
ing, still has not reached a satisfactory level, in 
Our opinion. 

Quite possibly the most basic cause of the 
schedule slippages, cost overruns, and dissatisfied 
users has been the errors that typically exist in re- 
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quirements statements. It is also quite possible 
that this condition has not been properly recog- 
nized by the technologists until fairly recently. 
Most of the attention on how to get improved sys- 
tems projects has been devoted to system design, 
programming, and data base design methods. It is 
finally becoming recognized that more attention 
must be given to ways for getting the require- 
ments right. 

The no-frills program that we have outlined for 
getting the requirements right, based on what we 
found in our study of the subject, does not seem to 
be too demanding. One change is in the frame of 
mind—simply acknowledging that numerous er- 
rors probably exist in any requirements state- 
ment. And most computer-using organizations 
have already obtained a degree of user in- 
volvement and have adopted methods for han- 
dling complexity. The changes that have costs 
attached to them are the software tools for pro- 
viding flexibility to change systems, the use of a 
formal inspection procedure, and the use of per- 
formance validation tests. 

“Getting the requirements right” is an area that 
deserves priority attention by data processing 
management. 
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